Method, apparatus, and non-transitory computer readable storage medium for confirming a perceived position of a traffic light

ABSTRACT

A method, apparatus, and computer-readable medium for confirming a perceived position of a traffic light, by obtaining identifiers and results of a first perception of traffic lights associated with the identifiers, the results of the first perception including a first estimation of an ellipse encompassing each of the traffic lights, receiving results of a second perception of traffic lights associated with the identifiers, the results of the second perception including a second estimation of an ellipse encompassing each of the traffic lights, calculating, based on the first perception and the second perception, association parameters for each possible pair of estimated ellipses, selecting, based on the calculated association parameters for each possible pair of estimated ellipses, matching pairs of estimated ellipses, and fusing each matching pair of estimated ellipses.

BACKGROUND Field of the Disclosure

The present disclosure relates to traffic light management andinteractions thereof with autonomous vehicles.

Description of the Related Art

At modern-day intersections, traffic lights and stop signs assist humandrivers in navigating their vehicles safely through cross traffic.However, as autonomous vehicles, wherein a computer is “behind thewheel”, become increasingly common, interactions between traffic lights,stop signs, and autonomous vehicles will become increasingly criticalfor driver and passenger safety.

While popular approaches utilize smart infrastructure to allowcommunication between vehicles and traffic light conditions, roadsignage, and the like, implementation of such strategies relies on thewidespread, and necessarily costly, deployment of smart devices. Assuch, a solution to providing awareness to autonomous vehicles, withoutthe need for system changes to traffic infrastructure, is needed.

The foregoing “Background” description is for the purpose of generallypresenting the context of the disclosure. Work of the inventors, to theextent it is described in this background section, as well as aspects ofthe description which may not otherwise qualify as prior art at the timeof filing, are neither expressly or impliedly admitted as prior artagainst the present disclosure.

SUMMARY

In an embodiment, the present disclosure further relates to a method forconfirming a perceived position of a traffic light, comprisingobtaining, by processing circuitry, traffic light identifiers andresults of a first perception of traffic lights associated with thetraffic light identifiers, the results of the first perception includinga first estimation of an ellipse encompassing each of the trafficlights, the first perception being performed by a first vehicle,receiving, by the processing circuitry, results of a second perceptionof traffic lights associated with the traffic light identifiers, theresults of the second perception including a second estimation of anellipse encompassing each of the traffic lights, the second perceptionbeing performed by a second vehicle, calculating, by the processingcircuitry and based on the obtained results of the first perception andthe received results of the second perception, association parametersfor each possible pair of estimated ellipses encompassing each of thetraffic lights perceived by the first vehicle and perceived by thesecond vehicle, each possible pair of the estimated ellipses includingone ellipse estimated by the first vehicle and one ellipse estimated bythe second vehicle, selecting, by the processing circuitry and based onthe calculated association parameters for each possible pair ofestimated ellipses, matching pairs of estimated ellipses, and fusing, bythe processing circuitry, each matching pair of estimated ellipses, eachfused matching pair corresponding to one of the traffic lightidentifiers.

In an embodiment, the present disclosure further relates to an apparatusfor confirming a perceived position of a traffic light, comprisingprocessing circuitry configured to obtain traffic light identifiers andresults of a first perception of traffic lights associated with thetraffic light identifiers, the results of the first perception includinga first estimation of an ellipse encompassing each of the trafficlights, the first perception being performed by a first vehicle, receiveresults of a second perception of traffic lights associated with thetraffic light identifiers, the results of the second perceptionincluding a second estimation of an ellipse encompassing each of thetraffic lights, the second perception being performed by a secondvehicle, calculate, based on the obtained results of the firstperception and the received results of the second perception,association parameters for each possible pair of estimated ellipsesencompassing each of the traffic lights perceived by the first vehicleand perceived by the second vehicle, each possible pair of the estimatedellipses including one ellipse estimated by the first vehicle and oneellipse estimated by the second vehicle, select, based on the calculatedassociation parameters for each possible pair of estimated ellipses,matching pairs of estimated ellipses, and fuse each matching pair ofestimated ellipses, each fused matching pair corresponding to one of thetraffic light identifiers.

In an embodiment, the present disclosure further relates to anon-transitory computer-readable storage medium storingcomputer-readable instructions that, when executed by a computer, causethe computer to perform a method for confirming a perceived position ofa traffic light, comprising obtaining traffic light identifiers andresults of a first perception of traffic lights associated with thetraffic light identifiers, the results of the first perception includinga first estimation of an ellipse encompassing each of the trafficlights, the first perception being performed by a first vehicle,receiving results of a second perception of traffic lights associatedwith the traffic light identifiers, the results of the second perceptionincluding a second estimation of an ellipse encompassing each of thetraffic lights, the second perception being performed by a secondvehicle, calculating, based on the obtained results of the firstperception and the received results of the second perception,association parameters for each possible pair of estimated ellipsesencompassing each of the traffic lights perceived by the first vehicleand perceived by the second vehicle, each possible pair of the estimatedellipses including one ellipse estimated by the first vehicle and oneellipse estimated by the second vehicle, selecting, based on thecalculated association parameters for each possible pair of estimatedellipses, matching pairs of estimated ellipses, and fusing each matchingpair of estimated ellipses, each fused matching pair corresponding toone of the traffic light identifiers.

The foregoing paragraphs have been provided by way of generalintroduction, and are not intended to limit the scope of the followingclaims. The described embodiments, together with further advantages,will be best understood by reference to the following detaileddescription taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings, wherein:

FIG. 1 is an illustration of an autonomous vehicle, according to anexemplary embodiment of the present disclosure;

FIG. 2 is a flow diagram of a method of confirming a perceived positionof a traffic light, according to an exemplary embodiment of the presentdisclosure;

FIG. 3 is an illustration of a covariance ellipse, according to anexemplary embodiment of the present disclosure;

FIG. 4A is an illustration of covariance ellipses when there is nooverlap between perceived positions of traffic lights, according to anexemplary embodiment of the present disclosure;

FIG. 4B is an illustration of covariance ellipses when there is overlapbetween perceived positions of traffic lights, according to an exemplaryembodiment of the present disclosure;

FIG. 5A is a flow diagram of a sub process of a method for confirming aperceived position of a traffic light, according to an exemplaryembodiment of the present disclosure;

FIG. 5B is an illustration of a sub process of resolving conflictsbetween perceived positions of traffic lights, according to an exemplaryembodiment of the present disclosure;

FIG. 6 is an illustration of fusion during a method for confirming aperceived position of a traffic light, according to an exemplaryembodiment of the present disclosure;

FIG. 7A is an illustration of a traffic light color estimation performedby a local vehicle, according to an exemplary embodiment of the presentdisclosure;

FIG. 7B is an illustration of traffic light color estimation fusion,according to an exemplary embodiment of the present disclosure;

FIG. 7C is an illustration of arbitration regarding traffic light colorestimation, according to an exemplary embodiment of the presentdisclosure;

FIG. 8 is a schematic of fleet distribution of a local navigational mapupdate, according to an exemplary embodiment of the present disclosure;

FIG. 9 is a schematic illustrating the communication architecture of asystem for generation of and fleet distribution of local navigationalmap updates, according to an exemplary embodiment of the presentdisclosure; and

FIG. 10 is a block diagram of a vehicle control system, according to anexemplary embodiment of the present disclosure.

DETAILED DESCRIPTION

The terms “a” or “an”, as used herein, are defined as one or more thanone. The term “plurality”, as used herein, is defined as two or morethan two. The term “another”, as used herein, is defined as at least asecond or more. The terms “including” and/or “having”, as used herein,are defined as comprising (i.e., open language). Reference throughoutthis document to “one embodiment”, “certain embodiments”, “anembodiment”, “an implementation”, “an example” or similar terms meansthat a particular feature, structure, or characteristic described inconnection with the embodiment is included in at least one embodiment ofthe present disclosure. Thus, the appearances of such phrases or invarious places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more embodiments without limitation.

In the not-so-distant future, city streets could be flooded withautonomous vehicles. Self-driving cars can move faster and travel closertogether, allowing more of them to fit on the road—potentially leadingto, as one example of many possible scenarios, congestion and gridlockon city streets.

Strategies for controlling traffic and enabling efficient travel ofautonomous vehicles include the use of vehicle to infrastructure (V2I)communication. V2I communication is the two-way exchange of informationbetween autonomous vehicles and traffic signals, lane markings, andother smart road infrastructure via a wireless connection. Generally,the goal of V2I communication is to improve road safety by reducingcollisions and supporting work zone and traffic management.

The use of V2I communication strategies, however, relies on thedeployment of a vast network of smart devices. For instance, V2Icommunication depends on smart traffic lights that are outfitted withcameras, communication units, and processing circuitry that enablesconveyance of traffic light status to approaching vehicles. Widespreadadoption of this approach can be impracticable when considered in viewof the cost and hierarchical control of infrastructure construction.

Accordingly, the present disclosure eliminates the need for V2Icommunication by implementing vehicle-based determinations that exploitvehicle to vehicle (V2V) communication. V2V communication provides amechanism for communication between vehicles that allows them todirectly share information about their position, speed, status, and thelike. For instance, in the context of the present disclosure, V2Vcommunication provides a means by which data generated by sensors (e.g.ultrasonic sensor, radar and camera technologies) of each vehicle can betransmitted to other vehicles for on-board processing.

In particular, the methods of the present disclosure provide a means bywhich an autonomous vehicle can confirm a position of a traffic light, astatus of the traffic light, and a relevance of the traffic light to theautonomous vehicle. This may be the case when several traffic lights arepresent at an intersection and an autonomous vehicle, in ordinarilydetermining a position of each traffic light, is unable to confidentlyassociate an observed traffic light with a position thereof.

According to an embodiment, the present disclosure provides a method bywhich a first vehicle may confirm an estimated position of a trafficlight. The confirmation may be performed by receiving relatedinformation from a second vehicle and, effectively, cross-referencingdata in order to identify and locate particular traffic lights.

In an embodiment, the confirmation performed during the methods of thepresent disclosure can be informed by additional data such as highdefinition (HD) maps and the like.

To this end, and with reference now to the figures, FIG. 1 provides anillustration of an autonomous vehicle having vehicle sensors configuredto perform sensing of the vehicle environment and, in particular,traffic guidance such as traffic lights.

FIG. 1 is an illustration of an autonomous vehicle (AV), according to anexemplary embodiment of the present disclosure. In order to operateaccurately and with precision, the AV 100 can be outfitted with aplurality of vehicle sensors, including, among others, one or morecameras 106, one or more surround view cameras 107, at least one radar(radio detection and ranging; herein “radar”) 108, at least one LiDAR(light detection and ranging; herein “lidar”) 109, at least oneultrasonic sensor 110, and one or more corner radar 111. Data acquiredfrom the plurality of vehicle sensors can be sent to a vehicle controlsystem 101, comprising, among other components, processing circuitry, astorage medium, image processing circuitry, and communication circuitry,in order to be processed, both locally and globally, and utilized innavigation. In one embodiment, the vehicle control system can be anelectronic control unit, “electronic control unit” being used therein todescribe any embedded system in automotive electronics that controls oneor more electrical systems or subsystems in a vehicle, including, amongothers, a telematics control unit and a powertrain control module. Oneimplementation of the vehicle control system is illustrated in FIG. 9 .

The above-described vehicle sensors of the AV 100 will be discussed inbrief below.

Regarding the one or more cameras 106, the cameras may be positionedalong a forward panel of the AV 100 and arranged such that, in the caseof a plurality of cameras, a parallax is created between the viewpoints.The parallax can be subsequently exploited, based upon the fixedgeometric relationship between the viewpoints along the panel of the AV100, to determine a distance to an obstacle, or impediment. To this end,the one or more cameras 106 may provide mono- or stereoscopicperspective. The one or more cameras 106 can employ, among othersensors, complementary metal-oxide-semiconductor (CMOS) image sensors.

Regarding the one or more surround view cameras 107, the cameras may bepositioned around the AV 100 in order to create a parallax and to obtaina 360° representation of the vehicle surroundings. As before, theparallax can be subsequently exploited, based upon the fixed geometricrelationship between the viewpoints, in order to determine a distance toan obstacle, or impediment. The one or more surround view cameras 107can employ, among other sensors, CMOS image sensors.

Regarding the above-described one or more cameras 106 and one or moresurround view cameras 107, in addition to distancing, the output of thecameras can be further processed by the vehicle control system 101 toidentify the vehicle surroundings. For instance, image processingcircuitry of the vehicle control system 101 can perform an imageclassification operation on the output of the cameras.

Regarding the at least one radar 108, the radar 108 may be positionedalong a forward panel of the AV 100. The at least one radar 108 can beone selected from a group of radars including, among others, short rangeradar, medium range radar, and long range radar. In an embodiment, andas employed commonly in Adaptive Cruise Control and Automatic EmergencyBraking Systems, the at least one radar 108 is long range radar, with anoperational range of, for example, a few hundred meters.

Regarding the at least one lidar 109, the lidar 109 may be positioned,for example, at a forward facing position and/or at a position with a360° viewpoint. The at least one lidar 109 can be an infrared lidarsystem using a rotating laser via a micro-electro-mechanical system, asolid-state lidar, or any other type of lidar. In one embodiment, the atleast one lidar 109 can provide a 905 nm wavelength with up to a 300meter operational range.

In an embodiment radar and lidar may be interchangeable for certaindistancing applications.

Regarding the at least one ultrasonic sensor 110, the ultrasonic sensor110 may be disposed at corners of the AV 100 for, in particular,short-range distancing. The at least one ultrasonic sensor 110 can be anultrasonic sensor having asymmetric directivity (110°×50°), shortringing time and high sound pressure, sensitivity and reliability, andbe configured to produce, among others, a 40 kHz, 48 kHz, 58 kHz, or 68kHz nominal frequency as required by the current situation.

Regarding the one or more corner radars 111, the corner radars 111 canbe substantially similar to the above-described at least one radar 108.Deployed as the corner radars 111, the one or more corner radars 111 canbe short range radar or medium range radar, as demanded, and can bebroadband Frequency Modulated Continuous Wave radar.

A combination of longitudinally acquired (time-based) data from theabove-described camera and distancing systems (radar and/or lidar) canbe used to extract speed and outlines of obstacles and moving objects.

According to an embodiment, and with reference to method 200 of FIG. 2 ,the above-described vehicle sensors, in communication with the vehiclecontrol system 101, allow for remote control of a driving operation ofthe AV 100. For example, the one or more cameras 106 can be used tooversee the surrounding of the AV 100 when the remote operator iscommanding the AV 100 to overtake an impediment. In instances where anobject is proximal to the AV 100, and wherein the above-describedcameras 106 are limited in their ability to accurately determinedistances, a remote operator may use additional sensors, as describedabove, such as the one or more surround view cameras 107, the radar 108,the lidar 109, the at least one ultrasonic sensor 110, and the at leastone corner radar 111, to remotely control the driving operation of theAV 100. It can be understood by one of ordinary skill in the art thatthe above-described vehicle sensors do not constitute an exhaustive listand are merely exemplary of vehicle sensors that may be found on an AV.In that context, any combination of vehicle sensors, described herein ornot, can be integrated in order to achieve the function of operation ofan AV, either autonomously or through remote operation.

Described herein, the AV 100 sensors detect traffic lights and performan association of the traffic lights from a high definition (HD) map byusing the distance uncertainty of the sensor (e.g., camera) detections.In an embodiment, the AV 100 can further include or be equipped with aninternet connection and can communicate with a host server. At least oneof the one or more cameras 106, the one or more surround view cameras107, the radar 108, the lidar 109, the at least one ultrasonic sensor110, and the at least one corner radar 111 can be configured to detectan attribute of a traffic light, such as a displayed color. The AV 100can further import auxiliary data, such as an HD map that includestraffic light attributes. For example, a live-updated map of a citytraffic light grid can be imported or evolved by the AV 100. Notably,the AV 100 can have a predetermined positional accuracy in the HD map,such as within 10 centimeters, or within 5 centimeters, or within 1centimeter of accuracy. The AV 100 can be configured to, via the vehiclecontrol system 101 (e.g., processing circuitry), perform an associationbetween the traffic lights in the HD map and a local perception aroundthe AV 100 detected by the many sensors.

To this end, FIG. 2 is a flow diagram of a method of confirming aperceived position of a traffic light, according to an exemplaryembodiment of the present disclosure.

In step 205, traffic light identifiers and results of a first perceptionof traffic lights associated with the traffic light identifiers areobtained, the first perception being performed by a first vehicle.

In step 210, results of a second perception of traffic lights associatedwith the traffic light identifiers are received, the second perceptionbeing performed by a second vehicle.

In step 215, association parameters for each possible pair of estimatedellipses encompassing each of the traffic lights perceived by the firstvehicle and perceived by the second vehicle are calculated.

In step 220, matching pairs of estimated ellipses based on thecalculated association parameters for each possible pair of estimatedellipses are selected.

In step 225, each matching pair of estimated ellipses are fusedtogether, each fused matching pair corresponding to one of the trafficlight identifiers.

FIG. 3 is an illustration of a covariance ellipse, according to anexemplary embodiment of the present disclosure. In an embodiment, amathematical representation is provided to model the traffic light erroras an ellipse to describe the distance uncertainty. The error indistance can be represented with what is called a covariance ellipseerror. In the special case of traffic light detection, this covarianceellipse can represent the error in range and bearing in polarcoordinates (r, θ). The covariance ellipse can be generated andprovided, and can be the result of a tracking process, such as a Kalmanfilter. The covariance ellipses can be represented by a semi-minor andmajor axis as the error in distance of a detected traffic light.

In an embodiment, the major axis is the range uncertainty, denoted asμ_(r), and bearing uncertainty, denoted as μ_(θ). The correspondingcovariance matrix is then defined as:

$R = \begin{bmatrix}\mu_{r} & 0 \\0 & \mu_{\theta}\end{bmatrix}$

Referring again to the figures, FIG. 4A is an illustration of covarianceellipses when there is no overlap between perceived positions of trafficlights, according to an exemplary embodiment of the present disclosure.In one example, no conflict exists between ellipses associations. Asshown, the AV 100 (via the above-described sensors and processingcircuitry included therein) can detect or identify a first covarianceellipse 191 associated with a first traffic light 11 and a secondcovariance ellipse 192 associated with a second traffic light 12. FIG.4A illustrates how, at the current position of the AV 100, both of thefirst traffic light 11 and the second traffic light 12 are not disposedalong a similar line of sight relative to the AV 100, and thus the firstcovariance ellipse 191 and the second covariance ellipse 192 aredistinguished by the AV 100 during an association process.

FIG. 4B is an illustration of covariance ellipses when there is overlapbetween perceived positions of traffic lights, according to an exemplaryembodiment of the present disclosure. In one example, conflict orconfusion can emerge during the association process of associating thefirst covariance ellipse 191 with the first traffic light 11 and thesecond covariance ellipse 192 with the second traffic light 12. Asshown, this can be due to the first traffic light 11 (and thus the firstcovariance ellipse 191) and the second traffic light 12 (and thus thesecond covariance ellipse 192) being disposed along a similar light ofsight relative to the AV 100. As such, the AV 100 may incorrectlydetermine there exists only one covariance ellipse (denoted as a mergedcovariance ellipse 191 m), and by association, only one traffic light.

By considering the aforementioned examples, when the AV 100 is onlyusing its local perception in FIG. 4B, the AV 100 would either associateboth of the first covariance ellipse 191 and the second covarianceellipse 192 to the first traffic light 11, or reject one, such as thesecond covariance ellipse 192, and associate the first covarianceellipse 191 to the first traffic light 11. The reason is that the firstcovariance ellipse 191 is closer to the second traffic light 12 than tothe first traffic light 11. Furthermore, the first covariance ellipse101 is also closer to the second traffic light 12 than the secondcovariance ellipse 192.

In order to solve association issues generated by the AV 100 localperception, V2V communication is used.

In an embodiment, the traffic lights in the HD map are assumed to beprecisely localized in the HD map. Once the AV 100 has identified thenext traffic lights on the route of the AV 100, the AV 100 will send tothe server the respective HD map traffic light IDs, and in addition, theresult of its local perception (traffic lights ellipses parameters(e.g., center in global coordinates, demi-axes, and angle)). By sendingthese unique traffic lights IDs, the AV 100 will automatically generatea request to the server that will send all the current availableinformation in a zone of the HD map around the target traffic lights.From the server perspective, the processing circuitry on the AV 100 willcollect all covariance ellipses from all the cars in a district or acity. Once the server receives the request from the AV 100, the serverwill send all the coordinates of the covariance ellipses of all roadusers proximal to or within a predetermined radius of the target trafficlight ID. The predetermined radius can be defined by a distance aroundthe target traffic light, such as 100 meters.

Described herein is a mathematical representation of the overlap of theellipses, such as the first covariance ellipse 191 and the secondcovariance ellipse 192. FIG. 5A is a flow diagram of a sub process of amethod for confirming a perceived position of a traffic light from theperspective of the AV 100, according to an exemplary embodiment of thepresent disclosure. In step 505, for each set of covariance ellipsesprovided by one vehicle, an association process is performed to matchpotential candidates from another set of ellipses provided by anothervehicle with respect to several criteria defined by a user. Thosecriteria can include, for instance, the Euclidian distance or theoverlap between ellipses. That is, the user can define or select thecriteria used during the association process, such as a Euclidiandistance-based association process or an ellipses overlap-basedassociation process. The selected criteria can be correlated to thequality of the traffic lights detection. In an embodiment, the criteriacan be selected by the AV 100.

In an event where the association is unclear, conflicts are resolved byassigning the best candidate with respect to the computed criteriadefined by the user.

To this end, FIG. 5B is an illustration of a sub process of resolvingconflicts between perceived positions of traffic lights, according to anexemplary embodiment of the present disclosure. FIG. 5B shows the AV 100and a second AV 102 along a same path and detecting the covarianceellipses of two traffic lights relative to their respective positions onsaid path.

In an example, the AV 100 can detect a first covariance ellipse 191 aand a second covariance ellipse 192 a, and the second AV 102 can detecta first covariance ellipse 191 b and a second covariance ellipse 192 b.

The possibilities of the second covariance ellipse 192 a are describedherein. In such an example, there is an overlap of 18% between thesecond covariance ellipse 192 a (detected by the AV 100) and the secondcovariance ellipse 192 b (detected by the second AV 102) with aEuclidean distance of 6.31 meters.

The possibilities of the first covariance ellipse 191 a are describedherein. In such an example, there is an overlap of 17% between the firstcovariance ellipse 191 a (detected by the AV 100) and the secondcovariance ellipse 192 b (detected by the second AV 102) with aEuclidean distance of 7.32 meters. In such an example, there is also anoverlap of 15% between the first covariance ellipse 191 a (detected bythe AV 100) and the first covariance ellipse 191 b (detected by thesecond AV 102) with a Euclidean distance of 5.31 meters.

The possibilities of the second covariance ellipse 192 b are describedherein. In such an example, there is an overlap of 18% between thesecond covariance ellipse 192 b and the second covariance ellipse 192 awith a Euclidean distance of 6.31 meters. In such an example, there isalso an overlap of 17% between the second covariance ellipse 192 b andthe first covariance ellipse 191 a with a Euclidean distance of 7.32meters.

The possibilities of the first covariance ellipse 191 b are describedherein. In such an example, there is an overlap of 15% between the firstcovariance ellipse 191 b and the first covariance ellipse 191 a with aEuclidean distance of 5.31 meters.

A conflict search process is described herein. The second covarianceellipse 192 b has two potential associations. A first associationincludes the overlap of 18% between the second covariance ellipse 192 band the second covariance ellipse 192 a, as well as the overlap of 17%between the second covariance ellipse 192 b and the first covarianceellipse 191 a. There is not a single way to resolve conflicts; in thedepicted example, conflicts are solved by considering the highest scorewith respect to the association criteria. In that case, the overlap ofthe second covariance ellipse 192 b and the second covariance ellipse192 a has a higher overlap and a shorter distance than that of thesecond covariance ellipse 192 b and the first covariance ellipse 191 a.As such, for the above-described example, the result is that the secondcovariance ellipse 192 a is associated with the second covarianceellipse 192 b, and the first covariance ellipse 191 a is associated withthe first covariance ellipse 191 b.

FIG. 6 is an illustration of fusion during a method for confirming aperceived position of a traffic light, according to an exemplaryembodiment of the present disclosure. In an embodiment, when all theellipses are processed and the conflicts resolved, the process stops.The result of ellipse association from the AV 100 and the second AV 102are then processed via a fusion process by the AV 100. In one example, aBayesian fusion is used. An example is provided as:

$\frac{1}{R_{fused}} = {\frac{1}{R_{vehicle1}} + \frac{1}{R_{vehicle2}}}$

The process can recursively be repeated for the next vehicles by the AV100, if any are present in a queue.

The AV 100 receives all the information of the equipped road vehicles(e.g., the second AV 102, a third AV, a fourth AV, etc.) around andcomputes the ellipses overlap between all the available ellipses,including the ones from its (the AV 100's) local perception, and fusesthe ellipses together. This is shown as a first fused covariance ellipse191 c and a second fused covariance ellipse 192 c, wherein the firstfused covariance ellipse 191 c represents the fusion associated with thefirst covariance ellipse 191 a and the first covariance ellipse 191 b,and the second fused covariance ellipse 192 c represents the fusionassociated with the second covariance ellipse 192 a and the secondcovariance ellipse 192 b.

After association, another problem to solve is determining a color ofthe light of the traffic light. A solution is described herein:

FIG. 7A is an illustration of a traffic light color estimation performedby the AV 100, according to an exemplary embodiment of the presentdisclosure. FIG. 7B is an illustration of traffic light color estimationfusion, according to an exemplary embodiment of the present disclosure.FIG. 7C is an illustration of arbitration regarding traffic light colorestimation, according to an exemplary embodiment of the presentdisclosure.

First, the AV 100 can detect the traffic lights and associate them tothe HD map traffic lights. When the color on the traffic light(s)changes, the colors and HD Map traffic light's unique IDs are sent tothe server (e.g., the cloud) for global fusion, which will consider allvehicles around this target area, such as a target intersection. Asshown in FIGS. 7A-7C, the traffic lights boxes are the result of thelocal fusion (performed by the AV 100) HD map traffic light associationcombined with color estimation and color estimation confidence level)and the demarcated points in FIGS. 7A-7C are the HD map traffic lightseach with its unique ID.

In FIG. 7B, the server can receive the traffic lights color from the AV100 and all vehicles and perform a fusion of the colors using thecollected, grouped information. The fusion process should be efficientin order to be reactive and inform the vehicles of the currentresult(s). A simple uniform probability association, for instance, issufficient.

EXAMPLE

In an example, the server can determine probabilities for traffic lightcolor(s) and select the color having the highest probability as thedetermined color of the traffic light. The result can be sent to the AV100 to perform the described local arbitration.

Traffic Light 1 (TFL1) and Traffic Light 2 (TFL2)—

TFL1 probabilities: RED 0.13 ORANGE 0.08 GREEN 0.79

TFL2 probabilities: RED 0.30 ORANGE 0.09 GREEN 0.61

Computation: RED(0.13+0.3)/2, (0.08+0.09)/2, (0.79+0.61)/2

Result=MAX(0.215,0.085,0.7)=0.7=GREEN

In FIG. 7C, the server can send back the result to the vehicles and theAV 100 performs an arbitration. This arbitration can be the maxprobability of the considered traffic lights.

TFL via the server: RED 0.2 ORANGE 0.01 GREEN 0.79

TFL via the AV 100: RED 0.01 ORANGE 0.91 GREEN 0.08

Result: ORANGE with 0.91 score

Notably, feedback from arbitration is feeding both processes(associating traffic lights and ellipses, and fusing overlappingellipses). Depending on the feedback, the processes can “rethink” theirstrategy without mixing local and global Traffic Lights.

Referring now to FIG. 8 , FIG. 8 is a schematic of fleet distribution ofa local NAV map update. According to an exemplary embodiment of thepresent disclosure, having updated a local NAV map for a single AV viavehicle ECU 858, the update can be sent to a cloud-computing environmentwherein the updated local NAV map can be integrated with the global NAVmap via server 860. In an embodiment, the updated local NAV map is alayer with discerning features relevant to only a region of the globalNAV map. In an embodiment, the updated local NAV map is a completeversion, or a complete version of a layer, of the NAV map that isintegrated with the global NAV map of the cloud-computing environment.

In an embodiment, a plurality of updated local NAV maps 859 can beconcurrently sent to the cloud-computing environment for integrationinto the global NAV map. The integrated global NAV map will be thecombination of all the local updates from all of the time-locked datafrom each AV.

In the end, having integrated the one or more updated local NAV mapsinto the global NAV map of the cloud-computing environment, the updatedglobal NAV map can be distributed to the fleet of AVs 820. Indistributing the updated global NAV map to each AV of the fleet of AVs820, each AV of the fleet of AVs 820 is then able to learn how to managean impediment as was done by a remote operator of the initial AV.Moreover, by learning how to manage the impediment or, for example,plurality of impediments at varying map zones, the fleet of AVs 820 canobviate the need to request remote operation when encountering those, orsimilar, impediments.

In an embodiment, sharing the updated global NAV map with each AV of thefleet of AVs 820 can be done in real-time or offline via wirelesscommunication from the cloud-computing environment.

In an embodiment, and in the case where multiple AVs are traveling alonga similar path, V2V communication can allow for more immediatecommunication of impediment circumnavigation.

For example, a situation may occur wherein a first AV, having utilized aremote operator in navigating an impediment, delays, according to aconfiguration of the first AV, updating a global NAV map until the firstAV returns to a parking spot, garage, or other ‘home’ location. In thiscase, the local NAV map update, and navigation therein, is not availableto subsequent AVs that may travel the same route prior to the first AVupdating the global NAV map (which, in turn, may be disseminated to thefleet of ‘subsequent’ AVs). V2V communication can be deployed in orderto transmit the local NAV map update from the first AV to proximate AVsthat may travel the same route. Therefore, though not updated within theglobal NAV map, the relevant remote operation path and annotations canbe distributed to proximate AVs.

Further to the above example, the updated local NAV map can bedistributed to, for instance, vehicles within a pre-determined radius ofthe first AV, the pre-determined radius being determined as a region oftravel wherein a subsequent AV is likely to traverse the same route asthe first AV. The pre-determined radius may be, assuming sufficientdensity of surrounding AVs to permit V2V communication, for instance, 10miles.

In the end, upon returning to the parking spot, the garage, or other‘home’ location, the first AV can transmit its local NAV map update tothe global NAV map.

With reference again to FIG. 8 and the vehicle ECU 858, a plurality oflocal NAV map updates, from a plurality of AVs, can be evaluated, eitherin a cloud-computing environment or on an AV, and merged into an updatedglobal NAV map. To this end, the global NAV map update can be handled indifferent ways.

First, if a local NAV map update is directed to a unique region or areaof the global NAV map, the local NAV map update can be simply mergedwith the global NAV map.

Second, if a local NAV map update is directed to a region or area alsoupdated by a second local NAV map update, the most recently recordedlocal NAV map update will be merged with the global NAV map.

Third, in a scenario where multiple local NAV map updates are recordedwithin a short time frame (e.g., concurrently or within minutes), thelocal NAV map update can be selected based upon the confidence level ofeach sensor and based upon the annotations of the remote operator.Accordingly, the most appropriate local NAV map update will be selectedby, for example, the cloud-computing environment, for merging with theglobal NAV map.

The confidence level of each sensor can be managed at two levels: (1)the level of the sensor and (2) the level of the remote operator. In anembodiment, (1) each sensor and sensor fusion on the AV can provide aconfidence level for the detection of an object and the classificationof an object. Accordingly, the vehicle control system can use thisinformation to compare concurrent local NAV map updates to determinewhich is more confident. Moreover, in an embodiment, (2) the remoteoperator can provide annotations as to the confidence level of theplurality of vehicle sensors based upon remote operator visualization inthe context of the performance of the plurality of vehicle sensors. Forexample, though a vehicle sensor may exhibit a high-level of confidence,a remote operator may determine that the vehicle control system has notaccurately detected and/or classified an object of the scene, and can,therefore, lower the confidence level of that vehicle sensor.

In an embodiment, and with regard to the third scenario described above,a situation may occur in which a first AV, having delayed transmittal ofan updated local NAV map, is followed by, for instance, a second AV anda third AV traveling along the same route which must also requestassistance from a remote controller. A distinct remotely controlled pathand annotation may then exist for each of the first AV, the second AV,and the third AV describing the same navigation. In one embodiment,evaluation and selection of the updated local NAV maps of the first AV,the second AV, and the third AV and merging with the global NAV map mayproceed, as described above, in the cloud-computing environment.

In an embodiment, selection of the updated local NAV maps of the firstAV, the second AV, and the third AV may be performed locally by thevehicle control system of one AV of the three AVs prior to beingtransmitted for merging with a global NAV map. For example, the first AVmay receive, upon returning to its parking spot, garage, or ‘home’location, the updated local NAV map from each of the second AV and thethird AV. Based upon the relative time of travel, the confidence levelof each sensor, and the annotations of the remote operator, and thelike, one of the three AVs may be selected, by the vehicle controlsystem of the first AV, to be transmitted and merged with the global NAVmap. The updated global NAV map can then be distributed to each AV ofthe fleet of AVs.

Having integrated the one or more local NAV map updates into the globalNAV map, and having shared the updated global NAV map with the fleet ofAVs, each AV of the fleet of AVs can learn how to handle the same, orsimilar, impediment, allowing the AV to proceed around the impedimentunder autonomous control.

In such instances, the vehicle can also use its own sensing technologyto understand the current environment, much in the same way as theoriginal impediment was detected. Based upon any changes, detected bythe plurality of vehicle sensors, in the current environment, thevehicle will react differently.

Initially, and as expected, if the vehicle control system, with theplurality of vehicle sensors, detects a similar environment or object tothe environment or object circumnavigated previously by a remoteoperator, the AV will use the trajectory provided by the remotelyoperated AV for navigation and, therefore, will not need the support ofthe remote operator.

However, as suggested, there may be changes in the environment. Forexample, if the vehicle control system, with the plurality of vehiclesensors, does not detect the ‘impediment’ previously circumnavigated andas expected from the local NAV map, the local NAV map can again beupdated according to the process described above, or can be reverted toa previous version of the local NAV map, if appropriate. Moreover,similarly to the above, the updated local NAV map can be sent to thecloud-computing environment, integrated with the global NAV map, anddistributed to the fleet of AVs for improved preparedness in navigation.

FIG. 9 illustrates an exemplary Internet-based navigation system,wherein AVs are connected to a remote operator and to a cloud-computingenvironment via waypoints that are connected to the Internet.

According to an embodiment, an AV 900 having a vehicle control system901 can connect to the Internet 980, via a wireless communication hub,through a wireless communication channel such as a base station 983(e.g., an Edge, 3G, 4G, 5G, or LTE Network), an access point 982 (e.g.,a femto cell or Wi-Fi network), or a satellite connection 981. Merelyrepresentative, each AV of a fleet of AVs 920 may similarly connect tothe Internet 980 in order to upload and download updated NAV maps. In anexample, a global NAV map can be stored in a data storage center 993 ofa cloud-computing environment 990. A cloud-computing controller 991 inconcert with a cloud-computing processing center 992 can permituploading, storing, processing, and downloading of NAV maps from thedata storage center 993. Updated local NAV maps can be transmitted tothe cloud-computing environment 990 via the Internet 980 for integrationwithin a global NAV map stored within the data storage center 993. Thecloud-computing processing center 992 can be a computer duster, a datacenter, a main frame computer, or a server farm. In one implementation,the cloud-computing processing center 992 and data storage center 993are collocated.

In an embodiment, raw and/or processed information from a plurality ofvehicle sensors can be transmitted to the cloud-computing environment990 for processing by the cloud-computing processing center 992 and/orstorage in the data storage center 993. In the case of raw information,the cloud-computing processing center 992 can perform processing similarto that performed by the vehicle control system 901 of the AV 900 duringAV operation. These processes include, among other processes, objectidentification and image classification.

According to an embodiment, a remote operator may perform annotation ona local NAV map at the level of the AV 900 or may perform annotation ona local NAV map at the level of the cloud-computing environment 990. Tothis end, a remote operator 956 can access the cloud-computingenvironment 990 through a remote control center 985 such as a desktop orlaptop computer or workstation that is connected to the Internet 980 viaa wired network connection or a wireless network connection.

FIG. 10 is a block diagram of internal components of an example of avehicle control system (VCS) that may be implemented, according to anembodiment. As discussed above, the VCS may be an electronics controlunit (ECU). For instance, VCS 1001 may represent an implementation of atelematics and GPS ECU or a video ECU. It should be noted that FIG. 10is meant only to provide a generalized illustration of variouscomponents, any or all of which may be utilized as appropriate. It canbe noted that, in some instances, components illustrated by FIG. 10 canbe localized to a single physical device and/or distributed amongvarious networked devices, which may be disposed at different physicallocations.

The VCS 1001 is shown comprising hardware elements that can beelectrically coupled via a BUS 1067 (or may otherwise be incommunication, as appropriate). The hardware elements may includeprocessing circuitry 1061 which can include without limitation one ormore processors, one or more special-purpose processors (such as digitalsignal processing (DSP) chips, graphics acceleration processors,application specific integrated circuits (ASICs), and/or the like),and/or other processing structure or means. The above-describedprocessors can be specially programmed to perform operations including,among others, image processing and data processing. Some embodiments mayhave a separate DSP 1063, depending on desired functionality. The VCS1001 also can include one or more input device controllers 1070, whichcan control without limitation an in-vehicle touch screen, a touch pad,microphone, button(s), dial(s), switch(es), and/or the like. The VCS1001 can also include one or more output device controllers 1062, whichcan control without limitation a display, light emitting diode (LED),speakers, and/or the like.

The VCS 1001 might also include a wireless communication hub 1064, whichcan include without limitation a modem, a network card, an infraredcommunication device, a wireless communication device, and/or a chipset(such as a Bluetooth device, an IEEE 802.11 device, an IEEE 802.16.4device, a Wi-Fi device, a WiMAX device, cellular communicationfacilities including 4G, 5G, etc.), and/or the like. The wirelesscommunication hub 1064 may permit data to be exchanged with, asdescribed, in part, with reference to FIG. 9 , a network, wirelessaccess points, other computer systems, and/or any other electronicdevices described herein. The communication can be carried out via oneor more wireless communication antenna(s) 1065 that send and/or receivewireless signals 1066.

Depending on desired functionality, the wireless communication hub 1064can include separate transceivers to communicate with base transceiverstations (e.g., base stations of a cellular network) and/or accesspoint(s). These different data networks can include various networktypes. Additionally, a Wireless Wide Area Network (WWAN) may be a CodeDivision Multiple Access (CDMA) network, a Time Division Multiple Access(TDMA) network, a Frequency Division Multiple Access (FDMA) network, anOrthogonal Frequency Division Multiple Access (OFDMA) network, a WiMAX(IEEE 802.16), and so on. A CDMA network may implement one or more radioaccess technologies (RATs) such as cdma2000, Wideband-CDMA (W-CDMA), andso on. Cdma2000 includes IS-95, IS-2000, and/or IS-856 standards. A TDMAnetwork may implement Global System for Mobile Communications (GSM),Digital Advanced Mobile Phone System (D-AMPS), or some other RAT. AnOFDMA network may employ LTE, LTE Advanced, and so on, including 4G and5G technologies.

The VCS 1001 can further include sensor controller(s) 1074. Suchcontrollers can control, without limitation, the plurality of vehiclesensors 1068, including, among others, one or more accelerometer(s),gyroscope(s), camera(s), RADAR(s), LiDAR(s), Ultrasonic sensor(s),magnetometer(s), altimeter(s), microphone(s), proximity sensor(s), lightsensor(s), and the like.

Embodiments of the VCS 1001 may also include a Satellite PositioningSystem (SPS) receiver 1071 capable of receiving signals 1073 from one ormore SPS satellites using an SPS antenna 1072. The SPS receiver 1071 canextract a position of the device, using conventional techniques, fromsatellites of an SPS system, such as a global navigation satellitesystem (GNSS) (e.g., Global Positioning System (GPS)), Galileo, Glonass,Compass, Quasi-Zenith Satellite System (QZSS) over Japan, IndianRegional Navigational Satellite System (IRNSS) over India, Beidou overChina, and/or the like. Moreover, the SPS receiver 1071 can be usedvarious augmentation systems (e.g., a Satellite Based AugmentationSystem (SBAS)) that may be associated with or otherwise enabled for usewith one or more global and/or regional navigation satellite systems. Byway of example but not limitation, an SBAS may include an augmentationsystem(s) that provides integrity information, differential corrections,etc., such as, e.g., Wide Area Augmentation System (WAAS), EuropeanGeostationary Navigation Overlay Service (EGNOS), Multi-functionalSatellite Augmentation System (MSAS), GPS Aided Geo Augmented Navigationor GPS and Geo Augmented Navigation system (GAGAN), and/or the like.Thus, as used herein an SPS may include any combination of one or moreglobal and/or regional navigation satellite systems and/or augmentationsystems, and SPS signals may include SPS, SPS-like, and/or other signalsassociated with such one or more SPS.

The VCS 1001 may further include and/or be in communication with amemory 1069. The memory 1069 can include, without limitation, localand/or network accessible storage, a disk drive, a drive array, anoptical storage device, a solid-state storage device, such as a randomaccess memory (“RAM”), and/or a read-only memory (“ROM”), which can beprogrammable, flash-updateable, and/or the like. Such storage devicesmay be configured to implement any appropriate data stores, includingwithout limitation, various file systems, database structures, and/orthe like.

The memory 1069 of the VCS 1001 also can comprise software elements (notshown), including an operating system, device drivers, executablelibraries, and/or other code embedded in a computer-readable medium,such as one or more application programs, which may comprise computerprograms provided by various embodiments, and/or may be designed toimplement methods, and/or configure systems, provided by otherembodiments, as described herein. In an aspect, then, such code and/orinstructions can be used to configure and/or adapt a computer (or otherdevice) to perform one or more operations in accordance with thedescribed methods, thereby resulting in a special-purpose computer.

It will be apparent to those skilled in the art that substantialvariations may be made in accordance with specific requirements. Forexample, customized hardware might also be used, and/or particularelements might be implemented in hardware, software (including portablesoftware, such as applets, etc.), or both. Further, connection to othercomputing devices such as network input/output devices may be employed.

With reference to the appended figures, components that can includememory can include non-transitory machine-readable media. The term“machine-readable medium” and “computer-readable medium” as used herein,refer to any storage medium that participates in providing data thatcauses a machine to operate in a specific fashion. In embodimentsprovided hereinabove, various machine-readable media might be involvedin providing instructions/code to processing units and/or otherdevice(s) for execution. Additionally, or alternatively, themachine-readable media might be used to store and/or carry suchinstructions/code. In many implementations, a computer-readable mediumis a physical and/or tangible storage medium. Such a medium may takemany forms, including but not limited to, non-volatile media, volatilemedia, and transmission media. Common forms of computer-readable mediainclude, for example, magnetic and/or optical media, a RAM, a PROM,EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier waveas described hereinafter, or any other medium from which a computer canread instructions and/or code.

The methods, systems, and devices discussed herein are examples. Variousembodiments may omit, substitute, or add various procedures orcomponents as appropriate. For instance, features described with respectto certain embodiments may be combined in various other embodiments.Different aspects and elements of the embodiments may be combined in asimilar manner. The various components of the figures provided hereincan be embodied in hardware and/or software. Also, technology evolvesand, thus, many of the elements are examples that do not limit the scopeof the disclosure to those specific examples.

Obviously, numerous modifications and variations are possible in lightof the above teachings. For example, the processes of the presentdisclosure may be performed entirely on the vehicle, entirely on theserver, or divided such that some are performed on the vehicle whileothers are performed on the server. Also, a vehicle may serve as aserver. It is therefore to be understood that within the scope of theappended claims, embodiments of the present disclosure may be practicedotherwise than as specifically described herein.

Embodiments of the present disclosure may also be as set forth in thefollowing parentheticals.

(1) A method for confirming a perceived position of a traffic light,comprising obtaining, by processing circuitry, traffic light identifiersand results of a first perception of traffic lights associated with thetraffic light identifiers, the results of the first perception includinga first estimation of an ellipse encompassing each of the trafficlights, the first perception being performed by a first vehicle,receiving, by the processing circuitry, results of a second perceptionof traffic lights associated with the traffic light identifiers, theresults of the second perception including a second estimation of anellipse encompassing each of the traffic lights, the second perceptionbeing performed by a second vehicle, calculating, by the processingcircuitry and based on the obtained results of the first perception andthe received results of the second perception, association parametersfor each possible pair of estimated ellipses encompassing each of thetraffic lights perceived by the first vehicle and perceived by thesecond vehicle, each possible pair of the estimated ellipses includingone ellipse estimated by the first vehicle and one ellipse estimated bythe second vehicle, selecting, by the processing circuitry and based onthe calculated association parameters for each possible pair ofestimated ellipses, matching pairs of estimated ellipses, and fusing, bythe processing circuitry, each matching pair of estimated ellipses, eachfused matching pair corresponding to one of the traffic lightidentifiers.

(2) The method according to (1), further comprising receiving, by theprocessing circuitry, the results of the second perception performed bythe second vehicle when it is determined there is overlap between theresults of the first perception of the traffic lights.

(3) The method according to either (1) or (2), further comprisingobtaining, by the processing circuitry, a first color status estimationfor each of the traffic lights associated with each fused matching pair,the first color status estimation being performed by the first vehicle,and receiving, by the processing circuitry, a second color statusestimation for each of the traffic lights associated with each fusedmatching pair.

(4) The method according to any one of (1) to (3), further comprisingperforming, by the processing circuitry and based on color statusestimations for each of the traffic lights associated with each fusedmatching pair, an arbitration to determine a likely color status foreach of the traffic lights.

(5) The method according to any one of (1) to (4), wherein the performedarbitration determines a color status having a maximum probability basedon a uniform probability association between color statuses and thecolor status estimations for each of the traffic lights.

(6) The method according to any one of (1) to (5), wherein each ellipseis a covariance ellipse and each estimation includes an estimation of acenter of the covariance ellipse in global coordinates, demi-axes, andangle.

(7) The method according to any one of (1) to (6), wherein theassociation parameters for each possible pair of estimated ellipsesinclude at least one of a Euclidean distance and a percentage of overlapbetween estimated ellipses of each possible pair of estimated ellipses.

(8) The method according to any one of (1) to (7), wherein the selectedmatching pairs of estimated ellipses are selected based on a comparisonof the association parameters for each possible pair of estimatedellipses and association criteria, the association criteria includingthresholds that values of the association parameters are compared to.

(9) The method according to any one of (1) to (8), further comprisingperforming, by the processing circuitry, the fusion of each matchingpair of estimated ellipses by Bayesian fusion.

(10) An apparatus for confirming a perceived position of a trafficlight, comprising processing circuitry configured to obtain trafficlight identifiers and results of a first perception of traffic lightsassociated with the traffic light identifiers, the results of the firstperception including a first estimation of an ellipse encompassing eachof the traffic lights, the first perception being performed by a firstvehicle, receive results of a second perception of traffic lightsassociated with the traffic light identifiers, the results of the secondperception including a second estimation of an ellipse encompassing eachof the traffic lights, the second perception being performed by a secondvehicle, calculate, based on the obtained results of the firstperception and the received results of the second perception,association parameters for each possible pair of estimated ellipsesencompassing each of the traffic lights perceived by the first vehicleand perceived by the second vehicle, each possible pair of the estimatedellipses including one ellipse estimated by the first vehicle and oneellipse estimated by the second vehicle, select, based on the calculatedassociation parameters for each possible pair of estimated ellipses,matching pairs of estimated ellipses, and fuse each matching pair ofestimated ellipses, each fused matching pair corresponding to one of thetraffic light identifiers.

(11) The apparatus according to (10), wherein the processing circuitryis further configured to receive the results of the second perceptionperformed by the second vehicle when it is determined there is overlapbetween the results of the first perception of the traffic lights.

(12) The apparatus according to either (10) or (11), wherein theprocessing circuitry is further configured to obtain a first colorstatus estimation for each of the traffic lights associated with eachfused matching pair, the first color status estimation being performedby the first vehicle, and receive a second color status estimation foreach of the traffic lights associated with each fused matching pair.

(13) The apparatus according to any one of (10) to (12), wherein theprocessing circuitry is further configured to perform, based on colorstatus estimations for each of the traffic lights associated with eachfused matching pair, an arbitration to determine a likely color statusfor each of the traffic lights.

(14) The apparatus according to any one of (10) to (13), wherein theperformed arbitration determines a color status having a maximumprobability based on a uniform probability association between colorstatuses and the color status estimations for each of the trafficlights.

(15) The apparatus according to any one of (10) to (14), wherein eachellipse is a covariance ellipse and each estimation includes anestimation of a center of the covariance ellipse in global coordinates,demi-axes, and angle.

(16) The apparatus according to any one of (10) to (15), wherein theassociation parameters for each possible pair of estimated ellipsesinclude at least one of a Euclidean distance and a percentage of overlapbetween estimated ellipses of each possible pair of estimated ellipses.

(17) The apparatus according to any one of (10) to (16), wherein theselected matching pairs of estimated ellipses are selected based on acomparison of the association parameters for each possible pair ofestimated ellipses and association criteria, the association criteriaincluding thresholds that values of the association parameters arecompared to.

(18) The apparatus according to any one of (10) to (17), wherein theprocessing circuitry is further configured to perform the fusion of eachmatching pair of estimated ellipses by Bayesian fusion.

(19) A non-transitory computer-readable storage medium storingcomputer-readable instructions that, when executed by a computer, causethe computer to perform a method for confirming a perceived position ofa traffic light, comprising obtaining traffic light identifiers andresults of a first perception of traffic lights associated with thetraffic light identifiers, the results of the first perception includinga first estimation of an ellipse encompassing each of the trafficlights, the first perception being performed by a first vehicle,receiving results of a second perception of traffic lights associatedwith the traffic light identifiers, the results of the second perceptionincluding a second estimation of an ellipse encompassing each of thetraffic lights, the second perception being performed by a secondvehicle, calculating, based on the obtained results of the firstperception and the received results of the second perception,association parameters for each possible pair of estimated ellipsesencompassing each of the traffic lights perceived by the first vehicleand perceived by the second vehicle, each possible pair of the estimatedellipses including one ellipse estimated by the first vehicle and oneellipse estimated by the second vehicle, selecting, based on thecalculated association parameters for each possible pair of estimatedellipses, matching pairs of estimated ellipses, and fusing each matchingpair of estimated ellipses, each fused matching pair corresponding toone of the traffic light identifiers.

(20) The non-transitory computer-readable storage medium according to(19), further comprising receiving the results of the second perceptionperformed by the second vehicle when it is determined there is overlapbetween the results of the first perception of the traffic lights.

Thus, the foregoing discussion discloses and describes merely exemplaryembodiments of the present disclosure. As will be understood by thoseskilled in the art, the present disclosure may be embodied in otherspecific forms without departing from the spirit thereof. Accordingly,the disclosure of the present disclosure is intended to be illustrative,but not limiting of the scope of the disclosure, as well as otherclaims. The disclosure, including any readily discernible variants ofthe teachings herein, defines, in part, the scope of the foregoing claimterminology such that no inventive subject matter is dedicated to thepublic.

The invention claimed is:
 1. A method for confirming a perceivedposition of a traffic light, comprising: obtaining, by processingcircuitry, traffic light identifiers and results of a first perceptionof traffic lights associated with the traffic light identifiers, theresults of the first perception including a first estimation of anellipse encompassing each of the traffic lights, the first perceptionbeing performed by a first vehicle; receiving, by the processingcircuitry, results of a second perception of traffic lights associatedwith the traffic light identifiers, the results of the second perceptionincluding a second estimation of an ellipse encompassing each of thetraffic lights, the second perception being performed by a secondvehicle; calculating, by the processing circuitry and based on theobtained results of the first perception and the received results of thesecond perception, association parameters for each possible pair ofestimated ellipses encompassing each of the traffic lights perceived bythe first vehicle and perceived by the second vehicle, each possiblepair of the estimated ellipses including one ellipse estimated by thefirst vehicle and one ellipse estimated by the second vehicle;selecting, by the processing circuitry and based on the calculatedassociation parameters for each possible pair of estimated ellipses,matching pairs of estimated ellipses; and fusing, by the processingcircuitry, each matching pair of estimated ellipses, each fused matchingpair corresponding to one of the traffic light identifiers.
 2. Themethod according to claim 1, further comprising receiving, by theprocessing circuitry, the results of the second perception performed bythe second vehicle when it is determined there is overlap between theresults of the first perception of the traffic lights.
 3. The methodaccording to claim 2, further comprising obtaining, by the processingcircuitry, a first color status estimation for each of the trafficlights associated with each fused matching pair, the first color statusestimation being performed by the first vehicle, and receiving, by theprocessing circuitry, a second color status estimation for each of thetraffic lights associated with each fused matching pair.
 4. The methodaccording to claim 3, further comprising performing, by the processingcircuitry and based on color status estimations for each of the trafficlights associated with each fused matching pair, an arbitration todetermine a likely color status for each of the traffic lights.
 5. Themethod according to claim 4, wherein the performed arbitrationdetermines a color status having a maximum probability based on auniform probability association between color statuses and the colorstatus estimations for each of the traffic lights.
 6. The methodaccording to claim 1, wherein each ellipse is a covariance ellipse andeach estimation includes an estimation of a center of the covarianceellipse in global coordinates, demi-axes, and angle.
 7. The methodaccording to claim 1, wherein the association parameters for eachpossible pair of estimated ellipses include at least one of a Euclideandistance and a percentage of overlap between estimated ellipses of eachpossible pair of estimated ellipses.
 8. The method according to claim 1,wherein the selected matching pairs of estimated ellipses are selectedbased on a comparison of the association parameters for each possiblepair of estimated ellipses and association criteria, the associationcriteria including thresholds that values of the association parametersare compared to.
 9. The method according to claim 1, further comprisingperforming, by the processing circuitry, the fusion of each matchingpair of estimated ellipses by Bayesian fusion.
 10. An apparatus forconfirming a perceived position of a traffic light, comprising:processing circuitry configured to obtain traffic light identifiers andresults of a first perception of traffic lights associated with thetraffic light identifiers, the results of the first perception includinga first estimation of an ellipse encompassing each of the trafficlights, the first perception being performed by a first vehicle, receiveresults of a second perception of traffic lights associated with thetraffic light identifiers, the results of the second perceptionincluding a second estimation of an ellipse encompassing each of thetraffic lights, the second perception being performed by a secondvehicle, calculate, based on the obtained results of the firstperception and the received results of the second perception,association parameters for each possible pair of estimated ellipsesencompassing each of the traffic lights perceived by the first vehicleand perceived by the second vehicle, each possible pair of the estimatedellipses including one ellipse estimated by the first vehicle and oneellipse estimated by the second vehicle, select, based on the calculatedassociation parameters for each possible pair of estimated ellipses,matching pairs of estimated ellipses, and fuse each matching pair ofestimated ellipses, each fused matching pair corresponding to one of thetraffic light identifiers.
 11. The apparatus according to claim 10,wherein the processing circuitry is further configured to receive theresults of the second perception performed by the second vehicle when itis determined there is overlap between the results of the firstperception of the traffic lights.
 12. The apparatus according to claim11, wherein the processing circuitry is further configured to obtain afirst color status estimation for each of the traffic lights associatedwith each fused matching pair, the first color status estimation beingperformed by the first vehicle, and receive a second color statusestimation for each of the traffic lights associated with each fusedmatching pair.
 13. The apparatus according to claim 12, wherein theprocessing circuitry is further configured to perform, based on colorstatus estimations for each of the traffic lights associated with eachfused matching pair, an arbitration to determine a likely color statusfor each of the traffic lights.
 14. The apparatus according to claim 13,wherein the performed arbitration determines a color status having amaximum probability based on a uniform probability association betweencolor statuses and the color status estimations for each of the trafficlights.
 15. The apparatus according to claim 10, wherein each ellipse isa covariance ellipse and each estimation includes an estimation of acenter of the covariance ellipse in global coordinates, demi-axes, andangle.
 16. The apparatus according to claim 10, wherein the associationparameters for each possible pair of estimated ellipses include at leastone of a Euclidean distance and a percentage of overlap betweenestimated ellipses of each possible pair of estimated ellipses.
 17. Theapparatus according to claim 10, wherein the selected matching pairs ofestimated ellipses are selected based on a comparison of the associationparameters for each possible pair of estimated ellipses and associationcriteria, the association criteria including thresholds that values ofthe association parameters are compared to.
 18. The apparatus accordingto claim 10, wherein the processing circuitry is further configured toperform the fusion of each matching pair of estimated ellipses byBayesian fusion.
 19. A non-transitory computer-readable storage mediumstoring computer-readable instructions that, when executed by acomputer, cause the computer to perform a method for confirming aperceived position of a traffic light, the method comprising: obtainingtraffic light identifiers and results of a first perception of trafficlights associated with the traffic light identifiers, the results of thefirst perception including a first estimation of an ellipse encompassingeach of the traffic lights, the first perception being performed by afirst vehicle; receiving results of a second perception of trafficlights associated with the traffic light identifiers, the results of thesecond perception including a second estimation of an ellipseencompassing each of the traffic lights, the second perception beingperformed by a second vehicle; calculating, based on the obtainedresults of the first perception and the received results of the secondperception, association parameters for each possible pair of estimatedellipses encompassing each of the traffic lights perceived by the firstvehicle and perceived by the second vehicle, each possible pair of theestimated ellipses including one ellipse estimated by the first vehicleand one ellipse estimated by the second vehicle; selecting, based on thecalculated association parameters for each possible pair of estimatedellipses, matching pairs of estimated ellipses; and fusing each matchingpair of estimated ellipses, each fused matching pair corresponding toone of the traffic light identifiers.
 20. The non-transitorycomputer-readable storage medium according to claim 19, furthercomprising receiving the results of the second perception performed bythe second vehicle when it is determined there is overlap between theresults of the first perception of the traffic lights.